Rate-Distortion Bound for Joint Compression and Classification

Rate-distortion theory is applied to the problem of joint compression and classification. A Lagrangian distortion measure is used to consider both the squared Euclidean error in reconstructing the original data as well as the classification performance. The bound is calculated based on an alternating-minimization procedure, representing an extension of the Blahut-Arimoto algorithm. As an example application, we consider a hidden Markov model (HMM) source, and the objective is to quantize the source outputs and estimate the underlying HMM state sequence (based on the quantized data). We present bounds on the minimum rate required to achieve desired average distortion on signal reconstruction and state-estimation accuracy.